Fourier Analysis and Filtering of aSingle Hidden Layer Perceptron

| We show that the Fourier transform of the linear output of a single hidden layer perceptron consists of a multitude of line masses passing through the origin. Each line corresponds to one of the hidden neurons and its slope is determined by that neuron's weight vector. We also show that convolving the output of the network with a function can be achieved simply by modifying the shape of the sigmoidal nonlinearities in the hidden layer. 1 Preliminaries Consider a layered perceptron with a single hidden layer of H hidden neurons. The output, (~ x), of the perceptron is linear and the input is a vector ~ x of dimension N. An example for the case N = 2, H = 3 is shown in Fig. 1(a). We have, for the general case: (~ x) = H X h=1 a h (~ w T h ~ x + h); (1) where ~ w T h = w h1 w h2 : : : w hN ], ~ x T = x 1 x 2 : : : x N ], T denotes transposition, and generally () is the sigmoid function, (z) = 1 1 + e ?z : (2) In the analysis to follow, however, (z) can be any function (e.g. Gaussian). Carroll & Dickinson (1989) showed that such networks can implement an arbitrarily good approximation to any L 2 function over ?1; 1] n. The functional form of the output of such a network was shown to be a nitely parameterized, approximate form of the back-projection operator, a component of the inverse Radon transform (Deans, 1983). Ito (1993) extended this result to cases where the domain of approximation is either a compact subset of the Euclidean space, or its entirety. Approximations of m-times continuously diierentiable functions in several variables, and their derivatives were considered. Our charter here is to examine the Fourier transform of the output of such a network. We assume that the neural network is trained and that all weights are xed. 2 Theory Deene the N dimensional Fourier transform (see for example (Marks, 1991)) of (~ x) as: